Skip to content

feat: parse Q10 archived map packets - #936

Open
hCoureau wants to merge 4 commits into
Python-roborock:mainfrom
hCoureau:feat/q10-archive-packets
Open

feat: parse Q10 archived map packets#936
hCoureau wants to merge 4 commits into
Python-roborock:mainfrom
hCoureau:feat/q10-archive-packets

Conversation

@hCoureau

@hCoureau hCoureau commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Summary

  • classify Q10 01/02/03/04 archive packet markers with a typed integer enum
  • parse saved-map and clean-record detail packet layouts, including bounded historical paths
  • route archive packets through the protocol and allow the map renderer to consume historical paths

Important

#933 has merged, and this branch now includes current main. This PR is the first remaining archive-stack change. Clean-record maps use Q10CleanRecordMapPacket, sharing the common grid fields and owning their historical path; current and saved-map packets do not carry that field. The renderer consumes the embedded path directly.

Design against the #933 review priorities

  • End-user API: no trait or end-user API changes are introduced in this slice.
  • Trait lifecycle: no trait state is introduced; archive consumption is deliberately deferred to feat: expose Q10 map archives #937.
  • Parser boundary: the protocol exposes typed map packets, with historical paths owned only by Q10CleanRecordMapPacket, so consumers do not inspect raw packet markers.
  • Parser internals: parsing is selected by the required typed packet kind and all archive-specific reads are bounded and covered by malformed/truncated-packet tests.

Review notes

This split incorporates the earlier review direction on #933: packet kinds own their wire integer values, the packet kind is required and parsed first, and unused trailing packet bytes are not exposed as public state.

No private map captures or account data are included.

Validation

  • 981 passed, including 92 snapshot tests
  • all pre-commit hooks passed
  • package build passed

Related work

AI assistance disclosure

This contribution was prepared with OpenAI Codex assistance. I reviewed the submitted changes and test results and take responsibility for the contribution.

Latest maintenance validation

  • Updated against upstream main at f58dfb2.
  • uv run pytest -q: 999 passed, including 92 snapshots.
  • uv run pre-commit run --all-files: passed.
  • uv build: sdist and wheel passed.
  • Maintenance and review fixes prepared with OpenAI Codex assistance; no new physical-device validation was performed for this update.

erase_zones = _parse_erase_zones(tail)
carpet_mask = _parse_carpet_mask(tail, width, height)
carpet_mask, carpet_end = _parse_carpet_block(tail, width, height)
if kind is Q10MapPacketKind.CLEAN_RECORD_DETAIL and carpet_end is not None:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when this kind of map packet is received, how many of these other fields are also included?

What i'm wondering is if a historical trace kind has a lot of overlap with other fields in Q10MapPacket or if it needs to be a separate type. basically as more fields are added the map packet seems like a sparse object. You could imagine each kind Q10MapPacketKind has a separate dataclass for example, if the overlap is low.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

re-reading the code I don't actually see the historical trace even being used anywhere yet, except in the tests.

Naively, it seems to me like we shouldn't be sticking this on to a map package and instead just using the kind to parse a clean record?

@allenporter allenporter left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am very happy to keep momentum going on this. I am adding additional feedback since this hasn't been moving since my last comment, in case it was missed. I have a lot of time to review during the us holidy weekend.

Comment thread roborock/protocols/b01_q10_protocol.py Outdated
Comment on lines +124 to +127
if Q10MapPacketKind.from_payload(payload) is Q10MapPacketKind.TRACE:
return parse_trace_packet(payload)
if Q10MapPacketKind.from_payload(payload) is not None:
return parse_map_packet(payload)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about we get the kind once:

Suggested change
if Q10MapPacketKind.from_payload(payload) is Q10MapPacketKind.TRACE:
return parse_trace_packet(payload)
if Q10MapPacketKind.from_payload(payload) is not None:
return parse_map_packet(payload)
kind = Q10MapPacketKind.from_payload(payload)
if kind is Q10MapPacketKind.TRACE:
return parse_trace_packet(payload)
if kind is not None:
return parse_map_packet(payload)

Comment thread roborock/map/b01_q10_map_parser.py Outdated


def lz4_block_decompress(data: bytes) -> bytes:
def lz4_block_decompress(data: bytes, max_output_size: int | None = None) -> bytes:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to make the second arg optional given we're updating all the callers in this change i believe.

virtual_walls: Sequence[Q10Zone] = ()


def render_q10_map(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Whats the relationship between a Q10MapPacket and a trace? especially Q10HistoricalTracePacket.

My impression is Q10HistoricalTracePacket is a field that was added to Q10MapPacket, so why would it also be passed here as trace?

Comment thread roborock/map/b01_q10_map_parser.py Outdated
carpet_mask = _parse_carpet_mask(tail, width, height)
carpet_mask, carpet_end = _parse_carpet_block(tail, width, height)
if kind is Q10MapPacketKind.CLEAN_RECORD_DETAIL and carpet_end is not None:
historical_trace, _ = _parse_clean_record_trace(tail, carpet_end)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we just ignore the second return value, so can it be removed?

erase_zones = _parse_erase_zones(tail)
carpet_mask = _parse_carpet_mask(tail, width, height)
carpet_mask, carpet_end = _parse_carpet_block(tail, width, height)
if kind is Q10MapPacketKind.CLEAN_RECORD_DETAIL and carpet_end is not None:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

re-reading the code I don't actually see the historical trace even being used anywhere yet, except in the tests.

Naively, it seems to me like we shouldn't be sticking this on to a map package and instead just using the kind to parse a clean record?

@hCoureau

Copy link
Copy Markdown
Contributor Author

@allenporter Thanks for the detailed feedback, and sorry I missed the earlier comments. Pushed cbd6abc and merged current main.

  • Clean-record maps now have a dedicated Q10CleanRecordMapPacket type; ordinary current/saved maps no longer carry historical_trace. The grid, rooms, erase zones, carpet and calibration are shared across all three map kinds, so those stay in the common base.
  • The renderer reads the historical path directly from the clean-record packet; callers only supply a separate trace for live maps. The consumer in feat: expose Q10 map archives #937 is updated too.
  • decode_message looks up the marker once, the LZ4 output bound is required, and the unused trace-parser return value is removed.

Validation: 999 tests and 92 snapshots passed, all pre-commit checks passed, and wheel/sdist builds passed. The dependent read-only PRs have been updated as well. Could you take another look when you have time, particularly at the shared-map/dedicated-clean-record type boundary?

These maintenance changes were prepared with OpenAI Codex assistance.

@allenporter allenporter left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for iterating on this and splitting out the clean-record path ownership.

Taking a close look at the resulting Q10CleanRecordMapPacket boundary, the **vars(packet) unpacking in parse_map_packet and the isinstance check in render_q10_map feel like an awkward fit for dataclass inheritance. Rather than subclassing Q10MapPacket, modeling this as a composite object Q10CleanRecordDetail(map=Q10MapPacket, trace=Q10HistoricalTracePacket | None) would be significantly cleaner and avoid both constructor duplication and isinstance branches in rendering.

I've also left a few inline comments on RoborockEnum fallback resilience, stripping stray origin points, and trimming internal wire fields from public models.

@classmethod
def from_payload(cls, payload: bytes) -> "Q10MapPacketKind | None":
"""Return the recognized kind for a payload marker."""
return next((kind for kind in cls if payload[:2] == kind.marker), None)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because this inherits from RoborockEnum, RoborockEnum._missing_ expects an unknown member (e.g. unknown = -1). Without it, _missing_ defaults to next(item for item in cls), which silently resolves unknown future wire markers to CURRENT = 1.

Also, rather than iterating all members with next((kind for kind in cls ...)) on every push, we can do an O(1) lookup:

class Q10MapPacketKind(RoborockEnum):
    """Semantic kind identified by a Q10 map packet's two-byte marker."""

    unknown = -1
    CURRENT = 1
    TRACE = 2
    CLEAN_RECORD_DETAIL = 3
    SAVED_MAP_DETAIL = 4

    @property
    def marker(self) -> bytes:
        """Return the two-byte wire marker for this packet kind."""
        if self is self.unknown:
            return b""
        return bytes((self.value, 1))

    @classmethod
    def from_payload(cls, payload: bytes) -> "Q10MapPacketKind | None":
        """Return the recognized kind for a payload marker."""
        if len(payload) < 2 or payload[1] != 1:
            return None
        kind = cls(payload[0])
        return None if kind is cls.unknown else kind

(Or if this enum is strictly an internal framing marker classifier, a standard IntEnum without RoborockEnum would avoid the fallback behavior altogether).



@dataclass
class Q10CleanRecordMapPacket(Q10MapPacket):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seeing **vars(packet) in parse_map_packet and test_b01_q10_render.py, along with isinstance(packet, Q10CleanRecordMapPacket) in render_q10_map, makes me wonder if subclassing Q10MapPacket is the right abstraction.

A clean-record response isn't really a specialized variant of a map raster; it's a compound payload containing both a map and a historical path. If we model it via composition instead:

@dataclass
class Q10CleanRecordDetail:
    map: Q10MapPacket
    trace: Q10HistoricalTracePacket | None = None

Then:

  1. Q10MapPacket remains pure and uniform across all map kinds (avoiding **vars(packet) or duplicating 9 constructor arguments).
  2. render_q10_map(packet, trace=None) remains pure and consumer-agnostic—callers just do render_q10_map(record.map, trace=record.trace) with zero isinstance branching or ignored arguments.


points: list[Q10Point] = field(default_factory=list)
version: int = 0
opaque_value: int = 0

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to expose version and opaque_value on the public dataclass?

version is already validated to equal 1 by the parser (returning None otherwise), and opaque_value is undecoded wire framing. Removing them keeps the public domain model clean and aligned with Q10TracePacket.

return Q10MapPacketKind.from_payload(payload) is Q10MapPacketKind.CURRENT


def is_clean_record_map_packet(payload: bytes) -> bool:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we prune is_clean_record_map_packet and is_saved_map_packet? Since decode_message now checks Q10MapPacketKind.from_payload(payload) directly, these single-expression helpers aren't used in production code.

Similarly, _parse_carpet_mask at line 711 appears to be an unused private compatibility helper that can be cleaned up.

return None
coordinates = struct.iter_unpack(">hh", memoryview(tail)[header_end:points_end])
return Q10HistoricalTracePacket(
points=[Q10Point(x=x, y=y) for x, y in coordinates],

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we run points = _drop_stray_leading_point(points) here as well?

If clean-record paths share the same firmware sentinel near (0, 0) that live traces have, leaving it in will cause visual lines to the origin and skew solve_q10_calibration.

Raises :class:`RoborockException` if map rendering fails.
"""
# An archived map owns its path; a live trace must never replace it.
render_trace = packet.historical_trace if isinstance(packet, Q10CleanRecordMapPacket) else trace

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line illustrates the friction with subclassing: render_q10_map takes trace: Q10TracePacket | None = None, but if packet is a Q10CleanRecordMapPacket, the passed trace argument is silently ignored.

If we use composition (Q10CleanRecordDetail(map=..., trace=...)), render_q10_map doesn't need this branch at all—the caller simply passes render_q10_map(record.map, trace=record.trace).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants